1. Technical Field
The present invention relates generally to quality assurance testing of computer systems and software.
2. Description of the Related Art
There are many well-known approaches to testing software during the development of a complex system. During the feature/test development phase, for example, a quality assurance engineer may devise a series of static test cases against which a system under development may be exercised. A test case is typically defined as a single step, and its expected result, along with one or more other pieces of information such as a case identifier, test author, or the like. Occasionally, a test case may be a series of steps with one expected result or outcome. The one or more steps associated with a given test case often are stored in a test plan document, a spreadsheet, a database, or other common repository (which may also store past test results and similar information). A set or collection of test cases is sometimes referred to a test suite.
The process of generating test cases is usually a manual one, although automated techniques have been described, e.g., U.S. Pat. No. 5,542,043. Regardless of the type of system involved, the test cases typically are derived from a well-defined series of simple discrete, atomic base actions (e.g., writing a set of files is a first action, reading the set of files is a second action, and so on). In this manner, and given a library of base actions, it is possible to design and implement a set of well-defined test cases. These test cases would then be useful to perform a given degree of test coverage for the system feature areas, but such testing is necessarily circumscribed by the nature and scope of the base library and the limited manner in which the actions may be associated with one another.
There are inherent issues with the prior art with respect to the veracity of a given test cases (e.g., the subjective value of the test case vis-à-vis the features actually tested), as well as the number of test cases that can be implemented efficiently. The latter problem typically is caused by time and labor restraints, especially where manual testing is used. While automated test case generation approaches (such as described in the above-identified patent) can be useful, they have not adequately addressed these problems.
It would be highly desirable to provide an improved technique for automating the generation and managed execution of a much broader range of test cases than has been possible using the prior art. The present invention addresses this problem.